[tune] Initial Commit for Tune CLI by richardliaw · Pull Request #3983 · ray-project/ray

richardliaw · 2019-02-07T23:38:28Z

This introduces a light CLI for Tune.

TODOs:

Make sure the columns are scoped to the terminal
Add documentation onto doc/source/tune-usage.rst
Verify that install enables tune ls

AmplabJenkins · 2019-02-08T00:09:16Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11667/
Test FAILed.

AmplabJenkins · 2019-02-09T04:14:54Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11715/
Test FAILed.

Tune CLI

AmplabJenkins · 2019-02-22T10:08:46Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12225/
Test FAILed.

AmplabJenkins · 2019-02-26T04:59:34Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12301/
Test FAILed.

python/ray/tune/scripts.py

python/ray/tune/trial_runner.py

AmplabJenkins · 2019-03-07T11:52:43Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12657/
Test FAILed.

hartikainen · 2019-03-07T20:19:46Z

$ tune list-trials ~/ray_results/gym/GoalHalfCheetah/v0/2019-03-03T19-19-32-goal-conditioned-sac-test-1/
Traceback (most recent call last):
  File "/home/kristian/conda/envs/softlearning/bin/tune", line 11, in <module>
    load_entry_point('ray', 'console_scripts', 'tune')()
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/kristian/github/hartikainen/ray/python/ray/tune/scripts.py", line 20, in list_trials
    commands.list_trials(experiment_path, sort)
  File "/home/kristian/github/hartikainen/ray/python/ray/tune/commands.py", line 145, in list_trials
    lambda t: datetime.fromtimestamp(t).strftime(TIMESTAMP_FORMAT))
  File "/home/kristian/conda/envs/softlearning/lib/python3.6/site-packages/pandas/core/series.py", line 3194, in apply
    mapped = lib.map_infer(values, f, convert=convert_dtype)
  File "pandas/_libs/src/inference.pyx", line 1472, in pandas._libs.lib.map_infer
  File "/home/kristian/github/hartikainen/ray/python/ray/tune/commands.py", line 145, in <lambda>
    lambda t: datetime.fromtimestamp(t).strftime(TIMESTAMP_FORMAT))
OverflowError: timestamp out of range for platform time_t

ipdb> l
    142     if "last_update_time" in checkpoints_df:
    143         from pprint import pprint; import ipdb; ipdb.set_trace(context=30)
    144
    145         datetime_series = checkpoints_df["last_update_time"].dropna()
    146         datetime_series = datetime_series.apply(
--> 147             lambda t: datetime.fromtimestamp(t).strftime(TIMESTAMP_FORMAT))
    148         checkpoints_df["last_update_time"] = datetime_series
    149
    150     if "logdir" in checkpoints_df:
    151         # logdir often too verbose to view in table, so drop experiment_path
    152         checkpoints_df["logdir"] = checkpoints_df["logdir"].str.replace(

ipdb> checkpoints_df
     trainable_name       ...        last_update_time
0  ExperimentRunner       ...                    -inf
1  ExperimentRunner       ...                    -inf
2  ExperimentRunner       ...                    -inf

[3 rows x 5 columns]

hartikainen · 2019-03-07T20:23:56Z

I did some testing with this. Overall it feels very neat, but there were some problems with the tune list-trials timestamp parsing (see the error above).

AmplabJenkins · 2019-03-07T23:57:07Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12668/
Test FAILed.

AmplabJenkins · 2019-03-08T03:31:52Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12680/
Test FAILed.

AmplabJenkins · 2019-03-08T04:34:57Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12681/
Test FAILed.

AmplabJenkins · 2019-03-08T07:57:01Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12688/
Test FAILed.

AmplabJenkins · 2019-03-08T12:07:56Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12699/
Test FAILed.

AmplabJenkins · 2019-03-08T12:27:02Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12700/
Test FAILed.

AmplabJenkins · 2019-03-08T21:55:35Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12704/
Test FAILed.

AmplabJenkins · 2019-03-08T23:10:01Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12710/
Test FAILed.

richardliaw · 2019-03-09T00:47:56Z

Jenkins tests passed before last commit, which was only a test change; relevant travis tests passed.

AmplabJenkins · 2019-03-09T01:57:36Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12714/
Test PASSed.

andrewztan · 2019-03-09T09:20:55Z

python/ray/tune/tests/test_commands.py

+    commands.list_experiments(project_path, info_keys=("total_trials", ))
+    captured = capsys.readouterr().out.strip()
+    lines = captured.split("\n")
+    assert sum("1" in line for line in lines) >= 3


should this be assert sum("1" in line for line in lines) >= num_experiments instead of 3

initial commit for tune cli

0fe4366

hartikainen and others added 5 commits February 7, 2019 18:12

Fix TrialRunner._start_time

5f1d5a0

Clean tune cli

d4a7b9a

Add explicit info keys for the experiment/project infos

400d382

small

65f718c

Merge pull request #10 from hartikainen/tune_cli

5b34ff3

andrewztan and others added 9 commits February 13, 2019 16:02

makes ls command print pretty table

3088016

makes ls command print pretty table

7a97f6a

Merge branch 'tune_cli' into tune_cli

5f44fc9

shortened output for logdir

e89fd3e

changed lsx to tabulate

c8acc11

added sorting for ls

e168652

fixed lsx and lint

e10425e

faster importing

6841a59

Merge pull request #12 from andrewztan/tune_cli

b759527

Tune CLI

richardliaw changed the title ~~[wip] [tune] Initial Commit for Tune CLI~~ [tune] Initial Commit for Tune CLI Feb 22, 2019

richardliaw added 2 commits February 25, 2019 17:52

Merge branch 'master' into tune_cli

9799c47

Merge branch 'tune_cli' of github.com:richardliaw/ray into tune_cli

c7d0f5c

andrewztan added 5 commits February 27, 2019 02:26

terminla resizing

63f6bdb

Merge branch 'tune_cli' of github.com:andrewztan/ray into tune_cli

05b6b56

lint

e238437

started cli docs

af9945c

Merge branch 'upstream_master' into tune_cli

71735e0

adizim reviewed Mar 1, 2019

View reviewed changes

python/ray/tune/scripts.py Outdated Show resolved Hide resolved

adizim reviewed Mar 1, 2019

View reviewed changes

python/ray/tune/trial_runner.py Outdated Show resolved Hide resolved

added docs

8e8b3db

Address comments and fix backward compat.

7b03b69

richardliaw added 3 commits March 7, 2019 17:00

Merge branch 'master' into tune_cli

6bb04b9

missed

db56b72

fix tests

ee1e48f

Fix tests

60a7672

richardliaw added 3 commits March 8, 2019 01:23

Merge branch 'master' into tune_cli

c097b45

travis

7309938

shorten

1cd9d3c

richardliaw added 4 commits March 8, 2019 10:56

fix py2

e512c63

Merge branch 'tune_cli' of github.com:richardliaw/ray into tune_cli

207e0f7

Merge branch 'master' into tune_cli

b33ff73

Fix trial

d8babd4

fix

739d9b4

richardliaw merged commit 6630a35 into ray-project:master Mar 9, 2019

andrewztan mentioned this pull request Mar 10, 2019

[tune] add-note command for Tune CLI #4321

Merged

andrewztan reviewed Mar 10, 2019

View reviewed changes

andrewztan mentioned this pull request Mar 10, 2019

[tune] add output flag for Tune CLI #4322

Merged

Conversation

richardliaw commented Feb 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AmplabJenkins commented Feb 8, 2019

Uh oh!

AmplabJenkins commented Feb 9, 2019

Uh oh!

AmplabJenkins commented Feb 22, 2019

Uh oh!

AmplabJenkins commented Feb 26, 2019

Uh oh!

Uh oh!

Uh oh!

AmplabJenkins commented Mar 7, 2019

Uh oh!

hartikainen commented Mar 7, 2019

Uh oh!

hartikainen commented Mar 7, 2019

Uh oh!

AmplabJenkins commented Mar 7, 2019

Uh oh!

AmplabJenkins commented Mar 8, 2019

Uh oh!

AmplabJenkins commented Mar 8, 2019

Uh oh!

AmplabJenkins commented Mar 8, 2019

Uh oh!

AmplabJenkins commented Mar 8, 2019

Uh oh!

AmplabJenkins commented Mar 8, 2019

Uh oh!

AmplabJenkins commented Mar 8, 2019

Uh oh!

AmplabJenkins commented Mar 8, 2019

Uh oh!

richardliaw commented Mar 9, 2019

Uh oh!

AmplabJenkins commented Mar 9, 2019

Uh oh!

andrewztan Mar 9, 2019

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

richardliaw commented Feb 7, 2019 •

edited

Loading